Stochastic Dynamics Research Articles

A simple theory for training response of deep neural networks

Abstract Deep neural networks give us a powerful method to model the training dataset’s relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a whole. The network’s behavior is training dynamics with a feedback loop from the evaluation of the loss function. We already know the training response can be constant or shows power law-like aging in some ideal situations. However, we still have gaps between those findings and other complex phenomena, like network fragility. To fill the gap, we introduce a very simple network and analyze it. We show the training response consists of some different factors based on training stages, activation functions, or training methods. In addition, we show feature space reduction as an effect of stochastic training dynamics, which can result in network fragility. Finally, we discuss some complex phenomena of deep networks.

Data-driven discovery of interpretable Lagrangian of stochastically excited dynamical systems

Exploring the intersection of deterministic and stochastic dynamics, this paper delves into Lagrangian discovery for conservative and non-conservative systems under stochastic excitation. Traditional Lagrangian frameworks, adept at capturing deterministic behavior, are extended to incorporate stochastic excitation. The study critically evaluates recent computational methodologies for learning Lagrangians from observed data, highlighting the limitations in interpretability and the exclusion of stochastic excitation. To address these gaps, an automated data-driven framework is proposed for the simultaneous discovery of Lagrange densities and diffusion coefficients of stochastically excited dynamical systems by leveraging sparse regression. This novel framework offers several advantages over existing approaches. Firstly, it provides an interpretable description of the underlying Lagrange density, allowing for a deeper understanding of system dynamics under stochastic excitations. Secondly, it identifies the interpretable form of the diffusion coefficient of generalized stochastic force, addressing the limitations of existing deterministic approaches. Additionally, the framework demonstrates robustness and versatility through numerical case studies encompassing both stochastic differential equations (SDEs) and stochastic partial differential equations (SPDEs), with results showing almost exact approximations to true system behavior and minimal relative error in derived equations of motion.

Updating Nonlinear Stochastic Dynamics of an Uncertain Nozzle Model Using Probabilistic Learning With Partial Observability and Incomplete Dataset

Abstract This article introduces a methodology for updating the nonlinear stochastic dynamics of a nozzle with uncertain computational model. The approach focuses on a high-dimensional nonlinear computational model constrained by a small target dataset. Challenges include the large number of degrees-of-freedom, geometric nonlinearities, material uncertainties, stochastic external loads, underobservability, and high computational costs. A detailed dynamic analysis of the nozzle is presented. An updated statistical surrogate model relating the observations of interest to the control parameters is constructed. Despite small training and target datasets and partial observability, the study successfully applies probabilistic learning on manifolds (PLoM) to address these challenges. PLoM captures geometric nonlinear effects and uncertainty propagation, improving conditional mean statistics compared to training data. The conditional confidence region demonstrates the ability of the methodology to accurately represent both observed and unobserved output variables, contributing to advancements in modeling complex systems.

DNA lesion bypass and the stochastic dynamics of transcription-coupled repair

DNA base damage is a major source of oncogenic mutations and disruption to gene expression. The stalling of RNA polymerase II (RNAP) at sites of DNA damage and the subsequent triggering of repair processes have major roles in shaping the genome-wide distribution of mutations, clearing barriers to transcription, and minimizing the production of miscoded gene products. Despite its importance for genetic integrity, key mechanistic features of this transcription-coupled repair (TCR) process are controversial or unknown. Here, we exploited a well-powered in vivo mammalian model system to explore the mechanistic properties and parameters of TCR for alkylation damage at fine spatial resolution and with discrimination of the damaged DNA strand. For rigorous interpretation, a generalizable mathematical model of DNA damage and TCR was developed. Fitting experimental data to the model and simulation revealed that RNA polymerases frequently bypass lesions without triggering repair, indicating that small alkylation adducts are unlikely to be an efficient barrier to gene expression. Following a burst of damage, the efficiency of transcription-coupled repair gradually decays through gene bodies with implications for the occurrence and accurate inference of driver mutations in cancer. The reinitation of transcription from the repair site is not a general feature of transcription-coupled repair, and the observed data is consistent with reinitiation never taking place. Collectively, these results reveal how the directional but stochastic activity of TCR shapes the distribution of mutations following DNA damage.

Dynamic linkages in agricultural and energy markets: A quantile impulse response approach

AbstractThis article investigates the dynamic linkages between agricultural and energy markets, with a focus on an econometric analysis of multivariate stochastic dynamics based on the joint distribution of state variables. The analysis relies on a quantile approach followed by the evaluation of a copula. Applied to nonlinear price dynamics, the approach is flexible and supports a general evaluation of impulse response functions representing how prices adjust over time and across markets in response to a given shock. The analysis allows for arbitrary distribution functions; it captures own‐price and cross‐price dynamics that can depend on the nature of shocks; and it also allows current changes to affect all moments of the future price distributions. The usefulness of the approach is illustrated in an econometric investigation of dynamic linkages in US corn, ethanol, and crude oil markets. We show how price adjustments can vary across quantiles, reflecting different speeds of adjustments depending on market conditions. We find evidence of nonlinear dynamics specific to the tails of the price distributions. We uncover evidence of positive contemporaneous codependence, especially tail dependence. We show how price shocks affect mean, variance, skewness as well as kurtosis of future price distributions. These results stress the importance of going beyond a standard mean‐variance analysis. They also shed new light on the deep linkages existing in the food‐fuel nexus.

Mott memristor based stochastic neurons for probabilistic computing

Many studies suggest that probabilistic spiking in biological neural systems is beneficial as it aids learning and provides Bayesian inference-like dynamics. If appropriately utilised, noise and stochasticity in nanoscale devices can benefit neuromorphic systems. In this paper, we build a stochastic leaky integrate and fire (LIF) neuron, utilising a Mott memristor’s inherent stochastic switching dynamics. We demonstrate that the developed LIF neuron is capable of biological neural dynamics. We leverage these characteristics of the proposed LIF neuron by integrating it into a population-coded spiking neural network and a spiking restricted Boltzmann machine (sRBM), thereby showcasing its ability to implement probabilistic learning and inference. The sRBM achieves a software-comparable accuracy of 87.13%. Unlike CMOS-based probabilistic neurons, our design does not require any external noise sources. The designed neurons are highly energy efficient and ultra-compact, requiring only three components: a resistor, a capacitor and a memristor device.

Stability analysis of stochastic differential systems with state‐dependent delay subject to average‐delay impulses

In this paper, we study the stochastic weak local exponential stability of stochastic differential systems with state‐dependent delay (SDD) subject to average‐delay impulses by using Lyapunov–Krasovskii functional and stochastic analytical skill. Unlike time‐dependent delay discussed in the previous literature, we introduce SDD in impulsive stochastic differential systems, where SDD is a stochastic variable associated with system state, consequently leading to an indeterminate delay boundary. Our findings reveal that when destabilizing delayed impulses satisfying specific conditions are generated, the stability of stochastic differential systems with destabilizing delayed impulses and stable continuous stochastic dynamics can still be maintained. Additionally, if stable delayed impulses satisfy certain conditions, stability of the systems under consideration can also be achieved, irrespective of the stable status on the continuous stochastic dynamics. Finally, two examples are given to demonstrate the validity of theoretical results.

Quickest Detection of Ecological Regimes for Natural Resource Management

AbstractWe study the stochastic dynamics of natural resources under the threat of ecological regime shifts. We establish a Pareto optimal framework of regime shift detection under uncertainty that minimizes the delay with which economic agents become aware of the shift. We integrate ecosystem surveillance in the formation of optimal resource extraction policies. We fully solve the case of a profit-maximizing monopolist, study its response to regime shift detection and show the generality of our framework by extending our results to other decision makers and functional forms. We apply our framework to the case of the Cantareira water reservoir in São Paulo, Brazil, and study the events that led to its depletion and the consequent water supply crisis.

Deterministic Versus Stochastic Dynamics of a Predator-Prey System with Disease in Prey: Impact of Fear and Supplementary Foods

Deterministic Versus Stochastic Dynamics of a Predator-Prey System with Disease in Prey: Impact of Fear and Supplementary Foods

Uniform propagation of chaos for a dollar exchange econophysics model

Abstract We study the poor-biased model for money exchange introduced in Cao & Motsch ((2023) Kinet. Relat. Models 16(5), 764–794.): agents are being randomly picked at a rate proportional to their current wealth, and then the selected agent gives a dollar to another agent picked uniformly at random. Simulations of a stochastic system of finitely many agents as well as a rigorous analysis carried out in Cao & Motsch ((2023) Kinet. Relat. Models 16(5), 764–794.), Lanchier ((2017) J. Stat. Phys. 167(1), 160–172.) suggest that, when both the number of agents and time become large enough, the distribution of money among the agents converges to a Poisson distribution. In this manuscript, we establish a uniform-in-time propagation of chaos result as the number of agents goes to infinity, which justifies the validity of the mean-field deterministic infinite system of ordinary differential equations as an approximation of the underlying stochastic agent-based dynamics.

Weak Pinning and Long-Range Anticorrelated Motion of Phase Boundaries in Driven Diffusive Systems.

We show that domain walls separating coexisting extremal current phases in driven diffusive systems exhibit complex stochastic dynamics with a subdiffusive temporal growth of position fluctuations due to long-range anticorrelated current fluctuations and a weak pinning at long times. This weak pinning manifests itself in a saturated width of the domain wall position fluctuations that increases sublinearly with the system size. As a function of time t and system size L, the width w(t,L) has a scaling behavior w(t,L)=L^{3/4}f(t/L^{9/4}), with f(u) constant for u≫1 and f(u)∼u^{1/3} for u≪1. An Orstein-Uhlenbeck process with long-range anticorrelated noise is shown to capture this scaling behavior. The exponent 9/4 is a new dynamical exponent for relaxation processes in driven diffusive systems.

Stochastic dynamics of mechanical systems with impacts via the Step Matrix multiplication based Path Integration method

AbstractIn this work we propose the Step Matrix Multiplication based Path Integration method (SMM-PI) for nonlinear vibro-impact oscillator systems. This method allows the efficient and accurate deterministic computation of the time-dependent response probability density function by transforming the corresponding Chapman–Kolmogorov equation to a matrix–vector multiplication using high-order numerical time-stepping and interpolation methods. Additionally, the SMM-PI approach yields the computation of the joint probability distribution for response and impact velocity, as well as the time between impacts and other important characteristics. The method is applied to a nonlinear oscillator with a pair of impact barriers, and to a linear oscillator with a single barrier, providing relevant densities and analysing energy accumulation and absorption properties. We validate the results with the help of stochastic Monte-Carlo simulations and show the superior ability of the introduced formulation to compute accurate response statistics.

Inhibiting presynaptic calcium channel motility in the auditory cortex suppresses synchronized input processing.

The emergent coherent population activity from thousands of stochastic neurons in the brain is believed to constitute a key neuronal mechanism for salient processing of external stimuli and its link to internal states like attention and perception. In the sensory cortex, functional cell assemblies are formed by recurrent excitation and inhibitory influences. The stochastic dynamics of each cell involved is largely orchestrated by presynaptic CAV2.1 voltage-gated calcium channels (VGCCs). Cav2.1 VGCCs initiate the release of neurotransmitters from the presynaptic compartment and are therefore able to add variability into synaptic transmission which can be partly explained by their mobile organization around docked vesicles. To investigate the relevance of Cav2.1 channel surface motility for the input processing in the primary auditory cortex (A1) in vivo, we make use of a new optogenetic system which allows for acute, reversable cross-linking Cav2.1 VGCCs via a photo-cross-linkable cryptochrome mutant, CRY2olig. In order to map neuronal activity across all cortical layers of the A1, we performed laminar current-source density (CSD) recordings with varying auditory stimulus sets in transgenic mice with a citrine tag on the N-terminus of the VGCCs. Clustering VGCCs suppresses overall sensory-evoked population activity, particularly when stimuli lead to a highly synchronized distribution of synaptic inputs. Our findings reveal the importance of membrane dynamics of presynaptic calcium channels for sensory encoding by dynamically adjusting network activity across a wide range of synaptic input strength.

Elastic particle model for coil-stretch transition of dilute polymers in an elongational flow

The phenomenon of the ‘coil-stretch’ (C-S) transition, wherein a long-chain polymer initially in a coiled state undergoes a sudden configuration change to become fully stretched under steady elongational flows, has been widely recognized. This transition can display intricate hysteresis behaviours under specific critical conditions, giving rise to unique rheological characteristics in dilute polymer solutions. Historically, microscopic stochastic models and Brownian dynamics simulations have shed light on the underlying mechanisms of the transition by uncovering bistable configurations of polymer chains. Following the initial work by Cerf (J. Chem. Phys., vol. 20, 1952, pp. 395–402), we introduce a continuum model in this study to investigate the C-S transition in a constant uniaxial elongational flow. Our approach involves approximating the unfolding process of the polymer chain as an axisymmetric deformation of an elastic particle. We make the assumption that the particle possesses uniform material properties, which can be represented by a nonlinear, strain-hardening constitutive equation to replicate the finite extensibility of the polymer chain. Subsequently, we analytically solve for the steady-state deformation using a polarization method. By employing this reduced model, we effectively capture the C-S transition and establish its specific correlations with material and geometric properties. The hysteresis phenomena can be comprehended through a force-balance analysis, which involves comparing the externally applied viscous forces with the intrinsic elastic responsive forces. We demonstrate that our model, while simple, unveils rich elastohydrodynamics of the C-S transition.

Stochastic Differential Equations with Singular Coefficients: The Martingale Problem View and the Stochastic Dynamics View

AbstractWe consider stochastic differential equations (SDEs) with (distributional) drift in negative Besov spaces and random initial condition and investigate them from two different viewpoints. In the first part we set up a martingale problem and show its well-posedness. We then prove further properties of the martingale problem, such as continuity with respect to the drift and the link with the Fokker–Planck equation. We also show that the solutions are weak Dirichlet processes for which we evaluate the quadratic variation of the martingale component. In the second part we identify the dynamics of the solution of the martingale problem by describing the proper associated SDE. Under suitable assumptions we show equivalence with the solution to the martingale problem.

Numerical treatment of stochastic and deterministic alcohol drinker dynamics with Euler–Maruyama method

Alcohol abuse is a substantial cause of various health and societal issues, as well as a significant factor in global disease. Once alcohol is consumed in the gastrointestinal tract, it undergoes metabolism in the liver and lungs. In this investigation, the nonlinear deterministic and stochastic differential frameworks are analyzed numerically to predict the dynamic evolution of the virus in the drinker alcohol model. The framework for apprehending drinking patterns is categorized into three distinct groups: the susceptible population, risk drinkers, and moderate drinkers. The approximate solution for each population group is determined by exhaustively creating scenarios that vary the probability ratio of infection in susceptible individuals who do not consume alcohol, the increasing rate of alcohol consumption, the rate at which individuals transition from acute to chronic drinking categories, the rate at which new non-drinking consumers are attracted, the death rate of the population, the ratio affecting the rate of sociability in heavy drinkers, and the overall population rate. The Euler–Maruyama approach for the stochastic framework and the Adams method for the deterministic framework are utilized, respectively, to determine the solutions of the alcohol drinker model. This study compares deterministic and stochastic frameworks to underscore their distinct characteristics and efficiency, achieved through comprehensive simulations and in-depth analysis of the numerical outcomes.

DP-GSGLD: A Bayesian optimizer inspired by differential privacy defending against privacy leakage in federated learning

Stochastic Gradient Langevin Dynamics (SGLD) is believed to preserve differential privacy as its intrinsic attribute since it obtain randomness from posterior sampling and natural noise. In this paper, we propose Differentially Private General Stochastic Gradient Langevin Dynamics (DP-GSGLD), a novel variant of SGLD which realizes gradient estimation in parameter updating through Bayesian sampling. We introduce the technique of parameter clipping and prove that DP-GSGLD satisfies the property of Differential Privacy (DP). We conduct experiments on several image datasets for defending against gradient attack that is commonly appeared in the scenario of federated learning. The results demonstrate that DP-GSGLD can decrease the time for model training and achieve higher accuracy under the same privacy level.

Stochastic Exchange Rate Dynamics, Intervention Dynamics and the Market Efficiency Hypothesis

AbstractSince currency price fluctuations hinder economic activity, exchange rate dynamics have an effect on national economies. To have a proper exchange rate policy in place, these dynamics are essential for nations with a trade economy. This study presents and examines a distinctive stochastic dynamics exchange rate model (ESI) in order to address the challenges associated with predicting the behavior of participants in some complex economic systems, which might lead to the system’s collapse. To address the issue of ESI stability by central bank interventions (managed currency) in a specified target value, a target value technique is also provided and tested. Last but not least, we examine the noise traders’ role as a major source of market uncertainty as we look at the market efficiency hypothesis for the foreign exchange market (FX).

Links synchronization control for complex networks with stochastic links dynamics

Links synchronization control for complex networks with stochastic links dynamics

Stochastic dynamics mass spectrometric structural analysis of poly(methyl methacrylate)

Polymers such as poly(methyl methacrylate), comprising of all-carbon backbones are regarded as ubiquitous functional materials mainly due to their low cost, unique physical properties, and impressive robustness. The latter properties are results from relative chemical inertness of the polymer backbone carbon–carbon bonds. The highlighted stability crucially challenges not only innovations to plastic recycling polymer materials, but also developments of reliable analytical protocols for determining both quantitatively and structurally polymers. Despite, the fact that there are numerous applications of mass spectrometric methods to polymer science, the analysis involves chiefly annotation of monomers of polymers and additives, owing to the fact that the additives are low molecular weight analytes. The exact structural determining of end-chain groups, particularly highlighting chemically substituted end carbon–carbon bond represents significant challenge even utilizing the superior features of the analytical mass spectrometric instrumentation. This study, in the latter context, illustrates innovative stochastic dynamics approach and model equations capable of not only determining unambiguously substituted carbon–carbon end chain of the entitled polymer but also to predict mass spectra of its derivatives: thus, extending crucially the applicability of the method to many fields of the fundamental scientific and industrial research. The latter claim is argued still at the beginning of the study with the achieved method performances showing |r|=0.99998. The study utilizes experimental and theoretical mass spectrometric data on high resolution electrospray ionization mass spectrometry; high accuracy quantum chemical static methods, molecular dynamics; and chemometrics.